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Summary 


Main Features 


Introduction and background 


1. The World Health Organisation (1980) has defined disability in the context of health 
experience as: ‘any restriction or lack (resulting from an impairment) of ability to perform an 
action in the manner or within the range considered normal for a human being’. Because it is 
dependent on respondent perceptions, disability is a difficult concept to measure even 
through special in-depth surveys involving personal interviews and a large number of 
questions. 


2. For planning, funding and delivery of community services, small geographic area 
estimates of people with a disability are required. There is also a demand for disability data 
for small population groups, such as Indigenous people. 


3. Since 1981, the Australian Bureau of Statistics (ABS) has conducted quinquennial 
surveys on disability. These provide widely-used and detailed information on the extent of 
disability in Australia, its effect on daily living and participation in the usual activities of the 
community, and the need for, and receipt of, assistance. However, survey information 
cannot be produced for small geographic areas or population groups. (Small area predictors 
have been modelled using survey, census and other data. However, these do not respond to 
local factors that affect the number of people with a disability in an area.). 


4. Consequently, there is a high level of demand for the 2001 Census to collect information 
on the prevalence of disability in the Australian population, to be used in conjunction with the 
more detailed survey data. The topic of disability attracted the largest number of 
submissions from users (31) in the last round of consultations about the content of the 2001 
Census, with strong support for disability status to be collected in subsequent censuses. 


5. The criteria for including a topic in the census are: 


e the topic is of major national importance 

e there is a need for data on the topic for small groups of the population or for small 
geographic areas 

e the topic is suitable for inclusion in the census. 


6. It is broadly agreed within the ABS that the first two criteria are met. However, there have 
been concerns about the third criterion because of the complex nature of defining and 
identifying disability, the limited space available on the census form and the method of form 
completion (self-enumeration by any household member). This report discusses the 


approach and results of the census testing. 
Testing 


7. Testing of a range of disability questions was undertaken in the lead up to the 1996 
Census. None was found to provide results sufficiently comparable to those obtained using 
surveys. 


8. A different approach to disability questions has been undertaken for the 2001 Census. 
Following discussions with the Census Consultative Group on Disability, two versions of 
questions were suggested, and tested in two rounds of focus group discussions in Sydney. 
Focus group participants included younger and older age groups, people with intellectual 
and psychiatric disability and their carers, and Indigenous people. Field testing of census 
questions (based on those recommended by the focus groups) was undertaken in Victoria 
during November 1997 and in South Australia during September 1998. 


9. The approach taken in both tests was to compare responses to the disability questions in 
the test census with the responses to a follow-up survey based closely on the questions 
used to establish disability in the ABS Survey of Disability, Ageing and Carers. In the census 
test, respondents were asked whether they experienced difficulties with a range of functions 
(e.g. hearing, living independently) and what caused the difficulties (e.g. age, disability, 
difficulty with English language). The disability questions used in each of the tests are 
shown in Appendix 1. 


10. Three approaches were trialled in the two census tests: 


e the November 1997 test used one form with a yes/no response for the difficulties 
question 

e the September 1998 test was split between two different forms, a yes/no response 
similar to the November 1997 test (form 4) and a scaled response (none/a little/a lot) 
to the difficulties question (form 5). Each form was administered to approximately 50% 
of the sample. 


11. The test areas had different profiles. The November 1997 test area had a higher 
concentration of disability pension recipients and hence a younger disability population. The 
September 1998 test area had a higher proportion of older people compared with the 
November 1997 test area and the general population. These factors contribute to the 
different rates of disability observed in the follow-up surveys. 


Results 


12. In assessing test outcomes a number of comparisons and indicators are relevant: 


e The closeness of the census and Survey disability rates - it is rarely possible to 
achieve the same precision in a census as a Survey. This is the case for all items in 
the census, not just disability. The census is a blunt instrument because of the need to 
use simpler questions appropriate to self-enumeration. To the extent to which 
differences are expected, the stability of the relationship between census and survey 
results is then an issue. 

e The proportion of false positives, i.e. those with disability in the census but not the 
survey. These should ideally be as low as possible. 

e The proportion of false negatives, i.e. those with disability in the survey but not the 
census. Experience in the development of disability surveys indicates that the more 
detailed and probing the questions, the greater the proportion of people with 


disabilities identified. Census questions would therefore be expected to produce a 
lower rate of disability than a survey. A greater proportion of false negatives than false 
positives would then be acceptable. 


13. The November 1997 test produced a census disability rate of 14.5 % compared to the 
survey rate of 23.1%. The disparity between these figures and the associated high false 
negative rate (48%) indicated that the census had only captured a part of the disability 
population. A second test was conducted in September 1998 to see if the questions could 
be improved. More specifically, its purpose was: 


e to test whether a different type of response to the census questions on the difficulties 
experienced with a range of functions improved the match with the total survey 
disability 

e to evaluate whether disability identified by the census questions had an acceptably 
close and stable relationship to an identifiable (more severely affected) subset of the 
survey disability population. 


14. The results of the three tests are summarised in the table below: 


Table 1: November 1997 and September 1998 Test Results 


November 1997 Test September 1998 Test 
Form type 4 Form type 5 
(yes/no response) (yes/no response) (scaled response) 
Census disability rate % 14.5 15.0 22.4 
Survey disability rate % 23.1 19.9 21.7 


False positives - 

census disability, survey no disability 

(% of census disability population) 17.4 30.9 36.0 
False negatives - 

survey disability, census no disability 

(% of survey disability population) 48.0 47.9 34.1 
Ratio of census to survey disability (%) 63.0 75.4 102.9 


15. The September 1998 test has narrowed the gap between the census and survey rates of 
disability at the expense of higher false positive rates. The fact that the 1998 test (Form type 
4) questions, which are only marginally different from the November 1997 test questions, 
lead to such different outcomes, underscores the difficulties inherent in measuring disability. 


16. One of the aims of the September 1998 test was to establish whether disability identified 
by the census questions had a relationship with an identifiable subset of the follow-up 
survey disability population. The scaled response questions in Form type 5 were included 
with this intent. The definitions of disability for the census and the survey were tightened (i.e. 
made more restrictive) to derive more severely affected subsets of the population with 
disabilities, and these populations were matched against each other (see Appendix 2). 


17. Results of this analysis for Form type 5 (Scaled response) are shown in Table 2. No 
improvements were achieved in matching the survey and census disability populations by 
restricting the disability definitions. In particular: 


e the differential between census and survey disability rates is greater for the more 


restricted definition than the broad one 


e the false positive rates remain high (30% and over) and the false negative rates 


increase with restrictions in the census definition, (to as high as 60%). 


Table 2: Results of Restricting Census and Survey Definitions - Form type 5 (scaled 


response)(a) 
Census disability definition - 
1 (broadest) 2 3 4 5 
Survey definitions - 
Any Moderate Severe restriction 
restriction restriction 

Census disability rate 24.8 22.4 18.8 7.9 7.9 
% 
Survey disability rate % 21.7 21.7 21.7 13.8 11.5 
False positives - 39.2 36.0 31.4 30.4 38.3 
census disability, 
survey no disability 
(% of census disability 
population) 
False negatives - 30.8 34.1 40.7 60.1 57.4 
survey disability, 
census no disability 
(% of survey disability 
population) 
Ratio of census to 113.9 102.9 86.4 57.3 69.0 


survey disability (%) 


(a) This table presents results of selected combinations of four different census disability definitions and three 
different survey disability definitions. See Appendix 2 for details. 


International experience 


18. Other countries also have a demand for disability information from their national 
censuses. They have experienced problems in achieving high quality data in a census, 
except for questions which were extremely limited in scope (e.g. missing limbs). The 
experience in other industrialised countries includes: 


e Canada (since 1986) and New Zealand (1996) have included disability questions to 
increase the efficiency of their disability sample surveys. Disability rates from their 
censuses have not been published. Canada will include a limited set of questions in 
their 2001 Census to serve as a screen for sample selection, but also to provide 


general information on the population reporting some form of disability. 


e The United Kingdom tested a disability question for their 1991 census but did not 
include the topic because of a high degree of non-response and poor data quality. 
They elected to include a question on long-term illness in place of disability, and 
published results from this question. 

e The most recent United States Census (1990) included some disability questions. 
Output identifies the proportion of people with work, mobility or self-care limitations. 


19. The experience of the above countries was that there was a high rate of people who 
appeared to have a disability (based on detailed testing), but were not classified as having a 


disability from the census questions. The reverse problem was also identified to a lesser 
extent (people who did not appear to have a disability but were classified as having a 
disability from the census questions). 


Table 3: Disability Tests in Other Countries 


Australia (Nov Australia (Sept 98) Canada NZ1995 UK 
97) : 1986 1991 


Form Form 
type 4 type5 


| 


Census disability rate % 14.5 15.0 22.4 9.2 19.7 6.8 
Survey disability rate % 23.1 19.9 21.7 12.0 35.6 12.6 
False positives - 

census disability, survey no 

disability 17.4 30.9 36.0 21.6 9.4 22.6 
(% of census disability 

population) 

False negatives - 

survey disability, census no 34.1 

disability 48.0 47.9 40.2 49.9 58.2 
(% of survey disability 

population) 

Ratio of census to survey 63.0 75.4 102.9 76.3 55.3 54.1 
disability (%) 


Conclusion 


20. While the results from the November 1997 census test raised some concerns about data 
quality of the disability item, they were broadly consistent with similar tests overseas. 


APPENDIX 1: Census Test Questions 


November 1997 Test 


For developing health policies and community- 
based programs, there is a need to measure the 
extent of disability existing in Australia 

14. For each of the following, tick YES or NO 

Does the person... 


© have difficulty in doing everyday activities such asO Yes O No 
eating, showering or dressing? 


e have difficulty talking to or hearing other people? O Yes O No 
e have difficulty learning or remembering things? O Yes O No 
© have difficulty reading or understanding things? O Yes O No 
© have difficulty walking, kneeling or climbing O Yes ONo 


stairs? 


© Have difficulty living in independent housing O Yes 


without help from other people? 


e Have difficulty doing any other things people of +O Yes 


the same age usually do? 


O No 


O No 


15. What causes the difficulty shown in Q14 9 Short-term health condition (lasting less than six months) 
O Long-term health condition 


for the person? 


O Disability 
O Age 


O Difficulty with English language 
O Other cause - please specify 


OO0O000000 
OO0000000 
OO0000000 
O No difficulty 


September 1998 Test 


Form type 4: Yes/no response 


For developing health policies and community- 
based programs, there is a need to measure the 


extent of disability existing in Australia 


18. Mark YES or NO for each of the following: 


Does the person have difficulty... 


© doing everyday activities such as eating, O Yes 


showering or dressing? 


e hearing? 


O Yes 


e learning, understanding or remembering things? O Yes 


© reading or seeing even with glasses? 
e walking, kneeling or climbing stairs? 


e living independently? 


O Yes 


O Yes 


O Yes 


e doing any other things people of the same age 0 Yes 
usually do (for example working, studying, etc.)? 


O No 


O No 


O No 


O No 


O No 


O No 


O No 


19. What causes the difficulty shown in Q18 9 Short-term health condition (lasting less than six months) 
O Long-term health condition 


for the person? 


O Disability 
O Age 


O Difficulty with English language 
O Other cause - please specify 


OO0000000 
OO0000000 
OO0000000 
O No difficulty 


September 1998 Test 


Form type 5: Scaled response 


For developing health policies and community-based 
programs, there is a need to measure the extent of 
disability existing in Australia 

18. How much difficulty does the person have in: 


None A little A lot 


© doing everyday activities such as eating, showering O Oo Oo 
or dressing? 


¢ hearing? oO o o 
e learning, understanding or remembering things? Oo Oo Oo 
® reading or seeing even with glasses? Oo Oo Oo 
® walking, kneeling or climbing stairs? Oo Oo Oo 
© living independently? Oo Oo Oo 
e doing any other things people of the same age Oo Oo Oo 


usually do (for example working, studying, etc.)? 


19. What causes the difficulty shown in Q18 9 Short-term health condition (lasting less than six months) 
for the person? O Long-term health condition 

O Disability 

O Age 

O Difficulty with English language 

O Other cause - please specify 

OOO0O00000 

OO0000000 

OO0000000 

O No difficulty 


APPENDIX 2: Census and Survey Disability Definitions 
Census definition 1 (broadest) 
Disability is identified if any of the following apply: 


e response to Q19 is ‘disability’ or ‘long-term condition’ 

e positive (yes/a little/a lot) response to Q18, Q19 is ‘age’, and age >60 

¢ positive response to Q18 ‘hearing’, conditional on age (>40) and no difficulty with 
English language 

e positive response to Q18 ‘everyday activities’, ' learning or understanding’, or ' doing 
things people of the same age do’, and age >9 

e positive response to Q18 ‘reading or seeing’, or ‘walking, kneeling, etc.', Q19 response 


is 'age’, and age >40 
3 or more positive responses to Q18, and Q19 response is ‘short-term condition’. 


Census definition 2 


Disability is identified if any of the following apply: 


response to Q19 is ‘disability’ or ‘long-term condition’ 

positive (yes/a little/a lot) response to Q18, Q19 is ‘age’, and age >60 
positive response to Q18 ‘hearing’, and no difficulty with English language 
positive response to Q18 ' learning or understanding’, and age >9. 


Census definition 3 


Disability is identified if any of the following apply: 


response to Q19 is ‘disability’ or ‘long-term condition’ 
positive (yes/a little/a lot) response to Q18, Q19 is 'age’, and age >60. 


Census definition 4 (most restricted) 


As for 2, but only for Form type 5, and using only ‘a lot' responses to Q18. 


Survey definitions 


Any restriction 


A person has a disability if he/she has one of the following: 


loss of sight (not corrected by glasses) 

loss of hearing, with difficulty communicating or use of aids 

loss of speech 

chronic or recurring pain that restricts everyday activities 

breathing difficulties that restrict everyday activities 

blackouts, fits or loss of consciousness 

difficulty learning or understanding 

incomplete use of arms or fingers 

difficulty gripping 

incomplete use of feet or legs 

a nervous or emotional condition that restricts everyday activities 

restriction in physical activities or physical work 

disfigurement or deformity 

needing help or supervision because of a mental illness or condition 

head injury, stroke or other brain damage, with long-term effects that restrict everyday 
activities 

treatment for any other long-term condition, and still restricted in everyday activities 
any other long-term condition that restricts everyday activities. 


Moderate activity restriction 


A person with a disability (as defined under any restriction above) who needed 
assistance, or had difficulty with: 


self care - bathing or showering, dressing, eating, using the toilet and managing 
incontinence 

mobility - moving around at home and away from home, getting into or out of a bed or 
chair, and using public transport 

communication - understanding and being understood by others: strangers, family and 
friends; and/or 

daily activities, such as health care, housework, home/garden maintenance, meal 
preparation, managing money/correspondence, and transport; and/or 

were restricted in schooling or employment. 


Severe activity restriction 


A person with a disability (as defined under any restriction above) who needed 
assistance with: 


e self-care mobility, communication; and/or 

e arange of daily activities; and/or 

e were restricted in schooling or employment (as specified in moderate activity 
restriction above). 


APPENDIX 3: Census Papers 


2001 Census Papers 


02/01 2001 Census: Digital Geography Technical Information Paper 
02/02 2001 Census: Report on Testing of Disability Questions for Inclusion in the 
2001 Census 


02/03 2001 Form Design Testing 


1996 Census Working Papers 


96/2 1996 Census Form Design Testing Program 


96/3 1996 Census of Population and Housing: Digital Geography Technical 
Information Paper 

97/1 1996 Census: Homeless Enumeration Strategy 

99/1 1996 Census: Industry Data Comparison 

99/2 1996 Census: Labour Force Status 

99/3 1996 Census Data Quality: Housing 

99/4 1996 Census: Review of Enumeration of Indigenous Peoples in the 1996 
Census 

99/6 1996 Census Data Quality: Occupation 

00/1 1996 Census Data Quality: Journey to Work 

00/2 1996 Census Data Quality: Qualification Level and Field of Study 

00/3 1996 Census Data Quality: Industry 

00/4 1996 Census Data Quality: Income 


For further information about these papers please contact Rosa Gibbs by telephone: (02) 


6252 5942 or email: rosa.gibbs@abs.gov.au. 


About this Release 


ABOUT THIS RELEASE 


Details testing that was undertaken on disability questions for inclusion in the 2001 Census. 
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